Delivering Educational Multimedia Contents through an Augmented Reality Application: a Case Study on Its Impact on Knowledge Acquisition and Retention

This paper presents a study to analyze the use of augmented reality (AR) for delivering multimedia content to support the teaching and learning process of the digestive and circulatory systems at the primary school level, and its impact on knowledge retention. Our AR application combines oral explanations and 3D models and animations of anatomical structures. A validation study was conducted with fourth grade students in order to evaluate the effect of our tool on knowledge retention. In this study, we attempt to verify whether students using the AR application retained more concepts that those learning the topic in a traditional setting. Results show an increased knowledge retention on students using AR multimedia contents as opposed to those following a traditional course, which validates AR technology as a promising tool to improve students’ motivation and interest, and to support the learning and teaching process in educational contexts. INTRODUCTION Nowadays, many educational institutions in developed countries are facing a lack of interest and motivation in students towards traditional academic practices. The growing distance between teaching procedures and the students’ technological way of life contributes to widen the gap. Up to the 19th century, formal education focused almost exclusively on lectures and recitations. Some early studies, however, explored the educational applications of manipulative interfaces. One of the first authors who studied what is commonly known as “hands-on learning” was Johann Heinrich Pestalozzi. He suggested that students learn best through their senses and through physical activity (Pestalozzi 1803). Today, some authors continue researching in this field. For example, Resnick et al. (1998) promoted children's learning based on manipulative interfaces, and established the idea that physical objects might play an important role in the learning process. It is still a relatively new idea in the history of education. O’Malley and StantonFraser (2004) discussed collaboration as a key factor for learning. Pontual and Price (2009) presented a motivational environment where students could learn the theory of light by experimenting with a tabletop with tangible elements. Other approximations can be found in the work of Fitzmaurice, Ishii and, Buxton (1995), Ishii and Ullmer (1997), and Ishii (2007), who proposed the use of tangible elements as tools to make interactions more natural and closer to the actions that take place in the real world. Motivation must be seriously considered because it is directly linked to learning achievements. Therefore, augmented reality (AR) applications, which are interactively and visually richer than traditional media, seem more attractive and motivating than traditional tools (Shelton and Hedley, 2002), (Duarte, Cardoso, and Lamounier, 2005). True learning requires experience. The more senses that are involved (sound, sight, touch, emotions, etc.), the more powerful the learning experience is. In this context, AR appears as an interesting emerging technology for education (Luckin and Fraser, 2011), (Lai and Hsu, 2011). AR combines real-time three-dimensional (3D) computer-generated models, video, and text superimposed onto real video-images. Several formal definitions and classifications for AR exist, including (Milgram, Kishino, 1994), (Milgram, Takemura, 1994). Azuma (1997) defined AR as a variation of virtual reality. Thus, AR supplements reality, rather than completely replacing it. With AR applications, it is possible to show the user a common space where virtual and real objects coexist in a seamless way. From a technological point of view, AR applications must fulfill the following three requirements (Azuma, 1997): combination of real and virtual worlds, real time interaction, and accurate 3D registration of virtual and real objects. As Billinghurst (2002) stated, although AR technology is not new, its potential in education is just beginning to be explored (Sumadio and Rambli, 2010). Some examples of AR applications in education can be found in Woods et al. (2004), who showed the educational benefits of virtual and augmented reality technology, particularly how these technologies improve the interpretation of spatial, temporal, and contextual content (Tettegah, Taylor, Whang, Meinstninkas, and Chamot 2006). Comparative studies between AR and traditional TOJET: The Turkish Online Journal of Educational Technology – October 2013, volume 12 issue 4 Copyright © The Turkish Online Journal of Educational Technology 20 classes (Kerawalla, Luckin, Seljeflot, and Woolard, 2006), (Freitas and Campos, 2008) have confirmed that AR enhances students learning. Moreover, some authors have suggested that AR technology improves kinesthetic learning because students interact directly with the educational material, associating the content with body movements and sensations (Seo, Kim and Kim, 2006). Although slower, this kind of learning allows improving retention of acquired knowledge. AR applications can be designed for different subjects and student levels. As an example, the Malaysian government recently created educational materials for a Road safety initiative, which included modules that combined mixed learning environments, interactive multimedia, AR and VR (Bakar, Zulkifli, and Mohamed, 2011). Other example is the work of Lin, Hsieh, Wang, Sie, and Chang (2011). In their project,the authors used AR and a touch-screen to enhance the educational resources about fish conservation in Taiwan. Their results focused on system usability, which was positive in an educational context. There are also a number of studies that explore the applications of AR technology to language learning. An interesting study was conducted by Ibáñez, Delgado, Leony, García, and Maroto (2011), where a multiuser AR platform for learning Spanish as a foreign language was developed. Results showed that AR has a positive effect on student motivation and improves the language learning process. Connolly, Stansfield, and Hainey (2011) developed an AR game for learning English as a foreign language to study how motivation could be improved through collaborative methods. 328 secondary school students and 95 language teachers from 17 European countries participated in this study. Most students and teachers were satisfied with the tools and expressed interest in learning other subjects using similar approaches. Other AR tools such as Construct3D (Kaufmann and Schmalstieg, 2003), have been specifically designed for Mathematics and Geometry education at high school and university levels, encouraging students to experiment with geometric constructions. Construct3D was validated as a simple learning tool which contributed significantly to the improvement of spatial abilities and to maximize learning transfer. VR and AR have also been used to provide an experience-based learning environment for understanding physics laws (Irawati, Hong, and Kim, 2008). In this system, simulation conditions were guaranteed via a 3D environment. Another AR application for Physics education was developed by Matsumoto, Miyauchi, Noguchi, and Yamashita (2012), where magnetic fields were visualized as realistic live magnetic distributions. Threedimensional renders and other virtual objects were also used to augment real objects in Chemistry (Chen, 2006). Virtual systems are useful when a laboratory is not available or when the experiences are dangerous, expensive, or time-consuming. Chen and Su (2011) conducted a study where elementary school children could learn to paint. The system used a sketch environment with computer vision and AR. Children could draw directly on the interface which provided additional functions such as contour extraction, image processing, and AR rendering. Results showed that the sketch system encouraged young children to participate and brought the natural painting experience to a virtual environment. The study opened up alternative opportunities for AR applications and tracking technologies. In a different study presented by Shamsuddin et al. (2010), Malaysian underwater habitats were simulated using AR. The virtual system provided similar educational value to students as that found in a real ocean, but time, cost, and manpower constraints were saved. As discussed in previous lines, AR in education has been used in every field of knowledge at every academic level, from kindergarten to college. Some AR studies specifically encourage the concept of manipulation learning, such as the one presented by Seo et al. (2006), who referred to this type of learning as hands-on of experience. Additionally, several European Union funded projects such as CONNECT (CONNECT, 2011), CREATE (CREATE, 2011) and ARISE (ARiSE, 2011) have designed and developed AR applications that provide good examples to make certain concepts easier to learn. In general, all the research mentioned previously evaluates students’ results and system usability in order to show improvement in the learning processes. System usability is a key factor to provide a successful learning experience, especially when manipulation is a significant part of the work presented in this paper. To evaluate this topic, different surveys have been suggested (Dumas and Redish, 1999). Since the participants of our study are children, a Likert scale (Albaum, 1997) was selected as a suitable tool to evaluate the system usability. This paper presents an AR system designed for fourth grade students to support the learning of the digestive and circulatory systems. Specific topics for the AR application were selected by a group of primary school teachers. The results of a comparative study show a significant increase in knowledge retention in students that used the TOJET: The Turkish Online Journal of Educational Technology – October 2013, volume 12 issue 4 Copyright © The Turkish Online Journal of Educational Technology 21 AR system over the ones that attended traditional master classes. Participants were highly motivated and expressed interest in using the new technology in the classroom. This paper is structured as follows: First, related work is presented. Next, a description of the system, interface design, and system architecture are introduced. Finally, the development procedure, validation protocol, results, and conclusions are discussed. MATERIALS AND METHODS Educational materials The system is comprised of the three AR applications presented in the ICALT 2010 conference (Pérez-López et al., 2010), and a new module, which increases its functionality. The role of the teachers involved in the development of the system was to support the creation of the tool by selecting the most appropriate materials to show the students. The materials accurately portrayed the digestive and circulatory systems and provided added value to the educational experience with respect to other strategies such as the use of videos and real animal organs. The system was organized into four applications because of the size of the 3D models used. The models are accessed by using a main menu with four buttons. The components and processes of the digestive and circulatory systems are explained in detail in each application. Different 3D models are rendered over the same AR marker and mixed with oral and textual explanations. In the first application, a naked human figure is initially displayed. When the user moves the AR marker close to the camera, the body figure becomes semitransparent, making the digestive system visible, which allows the user to see the different organs in detail. In addition, different sections, zoom levels and transparency levels can be applied to reveal internal organs, as shown in Figures 1and 2. Figure 1: Digestive system: mouth detail. Figure 2: Circulatory system: heart detail. In the second application, the digestive process of eating a cookie is illustrated: from the initial bolus formation in the mouth until the elimination of waste products and undigested materials. The third application is similar to the first one. In this case, the most important parts of the circulatory system are presented, as shown in Figure 3. In the fourth application, detailed visualizations of the heart movements, systemic and pulmonary circulation are presented (see Figure 4). Animations of the blood flow through vessels and lungs are also included. TOJET: The Turkish Online Journal of Educational Technology – October 2013, volume 12 issue 4 Copyright © The Turkish Online Journal of Educational Technology 22 Figure 3: Pulmonary circulation animation. Figure 4: Systemic circulation model. Interaction Design & Software Architecture The interface of the applications is comprised of several buttons located at either side of the screen to allow the selection of different modules, as shown in Fig. 3. Selections can be made by using the computer mouse or the AR marker. Selections with the AR marker are achieved by matching the position of the AR marker on the screen with the position of the buttons, as shown in Figure 5. Figure 5: Menu interaction using markers. Figure 6: System settings: user, screen, camera, marker, keyboard and mouse. Click events are simulated by maintaining the AR marker over the button for two seconds. The application uses the live streaming video captured by a web camera as a background, as shown in Figure 6. Additional interaction is provided by modifying the distance between the AR marker used to display the virtual objects and the webcam. A threshold is defined related to this distance which, when exceeded, triggers an event. This event is used to make the model transparent, show indicators with relevant names, or play audible and textual explanations over the model that is currently being shown (see Figure 7 and Figure 8). Both the transparency threshold and the model scale factor are configurable parameters. User preferences can be saved for future sessions. TOJET: The Turkish Online Journal of Educational Technology – October 2013, volume 12 issue 4 Copyright © The Turkish Online Journal of Educational Technology 23 Figure 7: Human body is shown when threshold is not reached. Figure 8: Human body becomes transparent exposing the digestive system when threshold is crossed. The system was developed using the game engine Conitec Gamestudio Pro A7 and our in-house AR software library, HumanAR (Martín-Gutiérrez et al., 2010). The first provides all the necessary tools for creating 3D and 2D games and real time graphics applications whereas HumanAR overcomes some drawbacks that are present in some existing AR libraries (reduced jitter, adaptive threshold to avoid illumination variations, infra-red marker detection, etc.). Validation Protocol As part of this paper, we performed a pilot study to find out the suitability of the AR application as a teaching and learning tool. To evaluate this, the validation protocol described below was applied. On one hand, the impact of the tool on knowledge acquisition and retention was evaluated. On the other hand, the interest and motivation generated by the AR application was studied. For our initial hypothesis, we stated that teaching using the AR tool provides a more effective learning experience than classical approaches based on anatomical illustrations and video sequences. If so, the performance and grades of the students using the AR tool should be higher than those using traditional learning methods. A preliminary study was conducted to analyze students’ grades that were obtained in other educational units in the “Knowledge of the natural, social and cultural environment” course. This analysis showed an average grade of 8.41 with a standard deviation of 1.46, which makes it difficult to show a significant improvement in these students’ performance (the Spanish grade system is based on a ten point scale, with 0 being the lowest and 10 the maximum grade). Therefore, we decided to evaluate the impact of the AR technology on knowledge retention (Kwon, Rasmussen, and Allen, 2005). Our new hypothesis states that the use of AR technology has a profound impact on students by, reinforcing the learned concepts. This means that after a certain period of time, students retain more concepts if AR contents were used during the learning process. To demonstrate this hypothesis, a quasi-experimental design of nonequivalent control groups was selected (Albaum, 1997). Although two different groups of fourth grade students were involved in our study, the collaborating school did not allow an entire class to be a control group. Reasons include the fact that the school is private, where parents are particularly involved in many decisions related to their children ́s education. In addition, we decided to give all children the opportunity to experience AR technology, especially after the school requested and financed part of the development. Because of the previous limitation in the experimental design, an alternative validation scenario was selected. All fourth grade students participated as a unique group on the study (two classes), and knowledge retention was chosen as the subject of analysis. In order to compare the effect of knowledge retention on participating students, teachers selected two educational units of similar complexity at their discretion: “Digestive system” and “Changes in the last century”. Both classes received one unit using the AR tool and the other unit was taught using traditional methods. The teachers remained unchanged for both educational units. The “Digestive system” unit was taught by the teachers using the AR tool, and then a questionnaire to evaluate tool usability was administered to the students. Next, learning achievements were evaluated using standard procedures (a written exam). A second assessment was performed two weeks later, without previous notice, and TOJET: The Turkish Online Journal of Educational Technology – October 2013, volume 12 issue 4 Copyright © The Turkish Online Journal of Educational Technology 24 finally, a third one, also without previous notice, four weeks after the initial one. The same methodology was used to assess the unit “Changes in the last century”, but in this case no AR contents were used. Each educational unit is taught during six 45 minute sessions. Regarding the unit "Changes in the last century", sessions 1, 2 and 3 were taught in the classroom, with the teacher lecturing in a traditional way. Students complete a series of activities in the textbook during the remaining three sessions. The dynamics of the "Digestive system” unit are as follows: the teacher presents the AR tool in session 1, i.e. how it works and how to use the AR markers. Each part of the digestive system is explained in detail using the AR tool. Session 2 is used to review the contents explained in the previous session, and to explain digestive processes using the animation tool. During session 3, previous contents are again reviewed, and students begin to use the AR tool individually. Finally, during sessions 4, 5 and 6, each student works in her own computer, performing the activities proposed by the unit, as shown in Figure 6. Motivation was measured by direct observation of children behavior and through a questionnaire. RESULTS 39 fourth grade students of two different classes participated in the validation study during a period of two months. This group consisted of 19 girls and 20 boys, between 9 and 11 years old (mean 10.03 and standard deviation 0.54). They played the roles of an experimental group and a control group, depending on the lesson, at the same time. Three traditional assessments for each lesson were proposed: the first one, as soon as lessons were taught; the second one, two weeks after the first assessment, without previous notice; and the third, four weeks after the first assessment, also without previous notice. Student grades for every assessment were ranged from 0 to 10. Our initial assessment for both lessons consisted of 23 questions, mostly short answer (one or two lines). The final question required students to write fifteen or twenty lines. Student grades were analyzed to estimate improvements in the learning process. Assuming a normal distribution in the initial data, a paired sample t-test (t Student), with a 95% confidence interval (ρ = 0.05) was applied. Our alternative hypothesis states that the grades obtained with the learning unit that was taught using AR are better than the grades obtained with the learning unit that was taught traditionally. As a null hypothesis, the differences between these grades were random, with no statistically significant differences between them. The results are shown in Table 1. Table 1: Initial assessment results AR learning, mean grade (std. dev.) Traditional learning, mean grade (std. dev.) t –Student (ρ = 0.05) p-value Initial assessment 6.34 (1.74) 6.31 (2.02) 0.11 0.916 As shown in Table 1, the p-value is not significant (p-value>0.05). As predicted, the analysis shows no statistically significant differences between the two methods during the initial assessment. A second assessment for both lessons was performed two weeks after the initial one. This time, with seven questions similar to the questions presented in the previous assessment were used, six questions which could be answered in a couple of lines and a final question that required a longer answer. The second assessment was intended to be easier than the initial one. At this point, we presumed that the impact of the AR technology would be noticeable. The same analysis with the same hypothesis was applied. The results are shown in Table 2. In this case, the p-value is significant (p-value<0.05), so the analysis shows statistically significant evidence that indicate that the grades obtained in the learning unit that was taught using AR are better than the ones obtained in the unit that was taught using traditional methods two weeks after the initial assessment. In addition, the effect size d was calculated to estimate the extent up to which the null hypothesis is false (Cohen, 1998). A value of 0.86 was obtained, which has a big effect according to Cohen (1998), (0.2 indicates a small effect; 0.5, a moderate effect; and 0.8, a big effect). Table 2: Retention assessment after two weeks AR learning, mean grade (std. dev.) Traditional learning, mean grade (std. dev.) t –Student (ρ = 0.05) p-value Effect size d Assessment after two weeks 6.77 (2.21) 4.89 (2.14) 6.17 0.000 0.86 TOJET: The Turkish Online Journal of Educational Technology – October 2013, volume 12 issue 4 Copyright © The Turkish Online Journal of Educational Technology 25 Finally, a third assessment was performed four weeks after the initial one for both learning units. Both of them consisted of twenty five true-false questions and a final long question where the student had to write her answer in fifteen or twenty lines. As the previous assessment, the impact of the AR technology should be noticeable, so again, the same analysis was performed. The new results are shown in Table 3. Table 3: Retention assessment after four weeks AR learning, mean grade (std. dev.) Traditional learning, mean grade (std. dev.) t –Student (ρ = 0.05) p-value Effect size d Assessment after four weeks 6.66 (2.26) 5.48 (1.93) 3.28 0.002 0.56 Once again, this new analysis provides a significant p-value. There are significant evidences that indicate that the grades obtained in the learning unit that was taught using AR are better than the grades obtained in the unit that was taught using traditional methods four weeks after the initial assessment. In this case, the effect size has a value of 0.56, which indicates a moderate effect according to Cohen (1998). Our study has demonstrated that knowledge retention increases if the content is delivered using AR. Additionally system usability and user motivation were also evaluated. The questionnaire shown in Table 4 was used. The questions were divided into two groups: the first five questions assess the class development using AR, and the next five questions evaluate AR as a tool. A five level Likert scale (Albaum, 1997) was used (1strongly disagree, 5-strongly agree). Table 4: Class and tool evaluation questionnaire